Adaptive sample size determination for the development of clinical prediction models

Abstract Background We suggest an adaptive sample size calculation method for developing clinical prediction models, in which model performance is monitored sequentially as new data comes in. Methods illustrate the approach using diagnosis of ovarian cancer ( n = 5914, 33% event fraction) and obstructive coronary artery disease (CAD; 4888, 44% fraction). used logistic regression to develop a consisting only priori selected predictors assumed linear relations continuous predictors. mimicked prospective patient recruitment by on 100 randomly patients, we bootstrapping internally validate model. added 50 random patients until reached 3000 re-estimated at each step. examined required satisfying following stopping rule: obtaining calibration slope ≥ 0.9 optimism c-statistic (or AUC) < 0.02 two consecutive sizes. This procedure was repeated 500 times. also investigated impact alternative modeling strategies: nonlinear correcting bias estimates (Firth’s correction). Results Better discrimination achieved (c-statistic with 7 predictors) than CAD 0.7 11 predictors). Adequate limited after median 450 (interquartile range 450–500) (22 events per parameter (EPP), 20–24) 850 (750–900) (33 EPP, 30–35). A stricter criterion, requiring AUC 0.01, met (23 EPP) 1500 (59 respectively. These sizes were much higher well-known 10 EPP rule thumb slightly recently published fixed Riley et al. Higher when relationships modeled, lower Firth’s correction used. Conclusions Adaptive determination can be useful supplement calculations, because it allows tailor specific context dynamic fashion.